Only recently has dynamic random-access memory (DRAM) become dense enough to be used for large data storage solutions. In addition, the volatility of DRAM has been reduced so that persistent random-access memory is readily available. These developments have resulted in solid-state drives (SSDs) being increasingly available and affordable as a storage solution to replace hard drives and optical drives.
Solid state drives contain non-volatile memory which can be read and written quickly since they do not require the seek time of traditional optical and magnetic drives. However, to obtain the full benefit of these faster data placement speeds, quicker bus connections are needed besides traditional SATA and SCSI connections. Thus, simple interface adapters have been created to connect the PCI Express ports of computers to solid state drives to increase the throughput.
Because SSDs have high throughput and faster read/write than other drives, they can be used both as auxiliary memory (RAM) for the CPU and for long-term storage replacing hard drives. These non-volatile memory express (NVMe) solutions are primarily for enhancing personal computers for gaming and data processing. NVMe connects to flash memory based SSDs via PCI Express. It was standardized around 2008 but did not reach consumer devices until later in 2013.
However, NVMe coupled with flash SSDs provides only a partial solution to hybrid storage. Specifically, the interface only provides a link between faster memory and generic computers or servers. The solution provides no stand alone benefits or capabilities besides the elimination of the bottleneck between CPU and SSD created by traditional bus interfaces.
For example, the NVRAM storage solution disclosed in US 2016/0308968 by Alex Friedman provides NVRAM cache as an intermediary between a host and long-term storage. This disclosure provides Redundant Array of Inexpensive Disks (RAID) segmentation on top of the SSDs and allows transfer of stripes of the RAID array to HDD storage. Thus, its use of SSD cache for the storage volume allows faster placement of data than traditional HDD storage, but provides no control, processing, or automation at the storage level. Rather Friedman's device relies on control servers separate from the host for all these decisions. Thus, the device is merely a variation on traditional storage solutions with faster SSD storage replacing a part of HDD storage.
Increasingly however, cloud computing solutions require smarter and more capable storage solutions. Just as super computers now combine massively parallel arrays of CPUs, GPUs, and/or ASICs into smart blocks with reconfigurable interconnects and distributed intelligence, a similar solution is needed for the storage side of a server. CPU arrays require additional memory besides that dedicated to their processing capabilities; likewise, GPUs and ASICs normally have little memory besides cache to handle the data awaiting processing.
Storage arrays have previously been developed for long-term storage of archive data, system back ups, and large data sets. Indeed many are so long term that the storage arrays often utilize automated tape drive storage to store the little-used data. Clearly seek times, bus bottlenecks, and data placement speeds are not of concern in these solutions.
Large data systems are usually composed of different variations of these storage arrays, called storage modules. These storage modules are managed in two ways: active-passive and active-active. Active-passive storage controllers are the most common implementation type for data storage. In this configuration, two (or more) controllers are installed in the storage array but the storage volumes are assigned to one controller. The second controller has access to the volumes, but is used only as a back-up if the first controller fails. If and when the first storage controller fails, the second controller seamlessly intervenes so that data access continues. An alternative to active-passive controllers is active-active controllers. With the active-active approach, all controllers are available to support all storage volumes and volumes are not assigned to any particular storage controller. If a controller fails, the surviving controllers can support the additional volumes from the failed controller.
The number of storage volumes or logically addressed partitions or logical unit numbers (LUNs) is usually quite large in a data management system. LUNs can cross server boundaries, hard drive boundaries, RAID boundaries, or re-divide those boundaries up in other ways. These LUNs need management, redundancy, and protection which are usually provided by a cluster management system. This management system will provision the LUNs and provide storage-area network (SAN) zoning and masking for the LUNs.
In addition, the storage processors manage the data placement and control on the various LUNs and/or storage modules via a Fibre Channel Protocol (FCP), iSCSI or the NVMe protocol, or both. The FCP/NVMe fabrics of physical interconnects carry SCSI commands to the LUNs over fiber, gigabyte Ethernet (GBE), or PCIe networks. The NVMe fabric additionally connects with SSD hard drive arrays via host bus adapters (HBAs).
Most storage management systems have several storage modules or several storage processors. Increasingly, the problem has become one of arbitration between different storage processors within the same active-active or active-passive cluster. Different application hosts access the storage cluster requesting download of data, processing of data, or storage of data. All these requests need arbitration between the storage processors and processing timeline monitoring. For example, if write and read requests are directed to the same data block by different storage processors within a short period of time, the read request will try to return data from the write request of the other storage processor even if it has not been placed yet.
Large data sets increasingly need to be available en masse for quick accessibility as various segments are compared, searched, merged, or sorted. Therefore, the volume of requests is exponentially increasing. Likewise, the size of the storage systems within a single cluster is increasing. As a result, large numbers of storage processors need continuous management and arbitration.